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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )M Responsive to communication(s) filed on 17 September 2004 . 
2a)S This action is FINAL. 2b)D This action is non-final. 

3) Q Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) M Claim(s) 1-26 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) I3 Claim(s) 1^26 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

, Application Papers 

9) D The specification is objected to by the Examiner. 

10) 13 The drawing(s) filed on 26 April 2001 is/are: a)[3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

11) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) 13 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

aMM\ b)D Some*c)D None of: 

1 .13 Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 

Attachment(s) 

1) □ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). . 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-1 52) 

3) □ Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 

U.S. Patent and Trademark Office 

PTO-326 (Rev. 04-01) Office Action Summary Part of Paper No. 0110 
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DETAILED ACTION 

1. Claims 1-26 have been presented for examination based on applicants 
amendment filed on 17 September 2004. Claims 1-26 remain rejected. 



Response to Arguments 

2. Applicants arguments filed on 17 September 2004 have been fully considered 
but are not persuasive. 

Regarding applicant's response to 112(1) rejection : Applicants now attempt to 
argue that the examiner has not established a prima facie case of non-enablement. 
Applicant's arguments center around the examiner's reference to the evaluation-and- 
control-unit's "automatic function". In this case applicants have completely 
misrepresented the examiner's rejection. The examiner's rejection of claims 1-12 is 
clearly drawn to the non-enablement of the evaluation-and-control-unit as recited below 
under 112(1) rejections and in the previous office actions. Specifically, applicant's 
specification discloses that the evaluation-and-control-unit uses an "automatic function " 
(specification: page 5, line 13) in selecting and identifying components. Hence, in 
looking to the specification for enablement for the claimed limitations relating to the 
evaluation-and-control-unit it would appear that the "automatic function" is an integral 
and critical part in the operation of the evaluation-and-control-unit . The examiner has 
therefor asserted that the specification's description of the "automatic function" is not 
sufficient to provide the necessary enablement for the claimed evaluation-and-control- 
unit . (see: 1 12(1) rejection below) Not that the "automatic function" has been rejected 
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as a claimed limitation under 112(1) as alleged by applicants. Merely stating that the 
"automatic function would easily be handled by a microprocessor running appropriate 
software", as alleged by applicants, does not address the enablement issue. The 
examiner therefor maintains that the specification does not provide a clear and concise 
description of the evaluation-and-control-unit. 

Applicants further argue that a skilled artisan would know how to implement the 
claimed features relating to "comparing information of installation components of a 
component library . with the picture data of the real installation to identify at least one of 
the installation components in the picture data as an identified installation component" 
by reading paragraphs [0009 - 0039] of application 2001/0025229. Specifically, that 
geometric attributes, e.g. size, shape, etc. with respect to components in the library and 
the stored picture data are compared in order to identify a matching installation 
component. The examiner has again reviewed these passages and respectfully 
disagrees. Applicants are claiming the ability "to identify at least one of the installation 
components in the picture data as an identified installation component" in claim 13. 
While, the recited passages make numerous vague references to using geometric 
properties in performing image analysis, none provide a clear and concise description 
the actual image analysis process. For example, paragraph [0036] recites, "the image 
analysis of the evaluation-and-control-unit attempts to match the geometric properties of 
the installation component with the geometric properties of the screen", however, it is 
inconceivable that a skilled artisan would be able to make and use the evaluation-and- 
control-unit from this description. There is no sufficient description of what the 
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geometric properties are, how they are matched, or how the image analysis actually 
"identifies" a component The examiner therefor maintains that the specification does 
not provide a clear and concise description of the subject matter claimed. 

In a nutshell, the specification does not disclose, and applicants response has 
not specifically indicated, specifically how the claimed limitations relating to deriving a 
hypotheses for identified components in picture data are realized, or how the 
evaluation-and-control-unit for comparing the component information data with 
picture data actually operates to "identify" the "installation component " sufficient to allow 
one skilled in the art to make and/or use the claimed invention without undue 
experimentation. Accordingly, the examiner maintains the 112(1) rejection. 

Regarding applicant's response to 103(a) rejections : Applicants first argue that 
the prior art (Hsu) does not teach the geometric method of image analysis disclosed by 
the present invention in paragraph [0030] and figure 1 of the specification. The 
examiner notes that, while the specific geometric image analysis process of the present 
invention has not been disclosed as noted above and below under 1 12(1) rejections, 
applicants arguments now rely on features which have not been specifically claimed. In 
response to applicant's argument that the references fail to show certain features of 
applicant's invention, it is noted that the features upon which applicant relies (i.e., 
geometric information contained in picture data and component information) are not 
recited in the rejected claim(s). Although the claims are interpreted in light of the 
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specification, limitations from the specification are not read into the claims. See In re 
Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). 

Applicants also appear to be engaging in piecemeal analysis by arguing, for 
example, that Hsu does not compare picture data to the actual installation, and that 
Crandall does not teach an evaluation-and-control-unit for comparing the component 
information data with picture data as specifically recited in the language of the claim. Hu 
teaches identifying components by comparing component information data (from a 
knowledge base) with real picture data, while Crandall teaches generating a virtual 
model of a real system using virtual components (installation components) as cited 
below under 103(a) rejections. Hence, claimed limitations are rendered obvious in view 
of the combination of Mark, Hu, and Crandall. One cannot show nonobviousness by 
attacking references individually where the rejections are based on combinations of 
references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & 
Co., 800F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). 

The examiner therefore maintains the 103(a) rejection of claims 1-26 as cited 

below. 

Regarding applicants comments on objection the specification on PTO-326 : 
Applicants are correct in assuming that the examiner has not objected to the 
specification. 

Regarding applicants comments on PTO-1449 references : Applicants are 
correct in assuming that the examiner has not considered the non-translated Japanese 
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language references crossed out on the PTO-1449. The examiner has however 
considered the corresponding US Patents as noted by applicants. 

Claim Rejections - 35 USC §112 

The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

3. Claims 1-12 are rejected under 35 U.S.C. 112, first paragraph, as containing 
subject matter which was not described in the specification in such a way as to 
enable one skilled in the art to which it pertains, or with which it is most nearly 
connected, to make and/or use the invention. 

Specifically, in independent claim 1 applicants are claiming an evaluation and 
control unit for comparing component information data with real picture data but the 
specification does not disclose an algorithm or technique for comparing component 
information data with real picture data. Neither the operation of the evaluation and 
control unit nor the process of identifying components in picture data, nor the 
claimed deriving hypotheses, is sufficiently described in the specification to allow one 
skilled in the art to make and/or use the invention . 

While page 4, line 5 of the specification states that the evaluation and control unit 
controls the process of generating the virtual installation model and controls an 
"automatic function " (page 5, line 13), it does not sufficiently describe functional 
operation either the process or the "automatic function". Obviously, one skilled in the art 
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would not be able to create an "automatic function" without specific details on how the 
automatic function operates. Merely stating that the evaluation and control unit 
"processes component data" (page 8, line 8) and "performs image analysis" (page 8, 
line 20) does not provide a sufficient description to allow one skilled in the art to make 
and/or use the invention. Dependent claims inherit these defects. 

Claims 13-26 are rejected under 35 U.S.C. 112, first paragraph, because the 
specification, while being enabling for "generating picture data" and "storage to 
store", does not reasonably provide enablement for "comparing picture data to 
identify installation components" (claim 13) or "a processing unit to compare 
components". The specification does not enable any person skilled in the art to 
which it pertains, or with which it is most nearly connected, to make and/or use 
the invention commensurate in scope with these claims. 

Specifically, independent claim 13 claims comparing picture data to identify 
installation components within the picture data but again the specification does not 
disclose an algorithm or technique for performing the comparison or identifying the 
components. 

Independent claim 25 claims a processing unit to compare components but 

does not disclose the process for comparing components sufficient to allow one skilled 
in the art to make and/or use the invention. 
Dependent claims inherit these defects. 
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Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103<a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 

USPQ 459 (1966), that are applied for establishing a background for determining 

obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

4. Claims 1-26 are rejected under 35 U.S.C. 103(a) as being unpatentable over 

U.S. Patent 6,025,847 issued to Marks in view of U.S. Patent 5.640.468 issued to 

Hsu in further view of U.S. Patent 5,552,984 issued to Crandall et al. 

While the specification for the claimed invention is delinquent in the areas cited 

above (see 112(1) rejections), the examiner has made prior art rejections based on the 

limited scope of the information contained in the specification. 

Independent claim 1 is drawn to: 

Generating an image of installation model by: 
Memory (1 st ) for storing picture data 
Memory (2 nd ) for storing component information 
Memory (3 rd ) for storing virtual installation model 

Evaluation and control unit for comparing component information data with real picture data 
Identifying components in picture data as installation components 
Deriving a hypotheses for identified components in picture data 
Generating respective installation components in virtual installation model 
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Regarding independent claim 1 : Marks teaches a system and method of 
generating a 3D (virtual) model from an image (picture) using a computer system. 
The image is of an arrangement of physical objects (components) where primitives 
(geometric) representing a portion of a physical object (a component) in the image are 
specified as a set of parameters that correspond locations in the image model. Marks 
further discloses incorporating a memory space for storing picture data, component 
(object) information. (Abstract, Summary of Invention, Figs. 1-7, CL3-L2-10, CL4-L45, 
CL7-L35-67, CL7-L5-50) 

Marks does not explicitly teach comparing component information data with real 
picture data. 

Hu teaches a system and method for identifying components by comparing 
component information data (from a knowledge base) with real picture data and 

incorporating multiple memories for storing picture data and component information. Hu 
further discloses evaluating (deriving a hypotheses via the image system) identified 
components (objects) in the picture data. (Abstract, Summary of Invention, Figs. 1-4, 6, 
8a-10b, CL13-L15-CL14-55) 

Hu mentions, but further does not explicitly teach virtual components 
(installation). 

Crandall teaches generating a virtual model of a real system using virtual 
components (installation components) from a library of components representing the 
total virtual system (installation). (Abstract, Summary of Invention, Figs. 1a&b, 9, CL2- 
L5-11, CL4-L1-29) 
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It would have obvious to one of ordinary skill in the art at the time the claimed 
invention was made to modify the teachings of Marks relating to a system and method 
for generating a 3D (virtual) model from an image (picture), with the teachings ofHu 
relating to identifying components by comparing component information data with 
real picture data from a knowledge base, and to further modify the teachings of Marks 
with the teachings of Crandall relating to generating a virtual model of a real system 
using virtual components (installation components) from a library of components 
representing the total virtual system (installation), to realize a device and method for 
generating a virtual model of an installation. An obvious motivation exists since, as 
referenced by prior art, creating a 3D (virtual) model of physical objects provides more 
efficient detection of discrepancies between the model and the actual image. (See 
Marks Background) 

Regarding dependent claims 2-4 : Hu performs image analysis on picture data as 
previously cited above. Crandall teaches using virtual components (installation 
components) from a library of components representing the total virtual system 
(installation) (CL2-L5-11). Marks teaches objects being located by geometric 
information and multiple window views of picture data and 3D (virtual) model views. 
(CL4-L10-25, Figs. 2-7) 

Regarding dependent claims 5-9 : Marks discloses building a 3D (virtual) model 
using the well-known techniques of "dropping", "clicking and dragging", and "rubber 
banding" in the manipulation of primitives based on geometric information (CL4-L42). 
Marks also teaches the evaluation of structural components (by function) to assign 
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primitives (add components) in a 3d (virtual) installation model. (Figs. 9-25, CL3-L29-43, 
CL5-L12-25, CL6-L10-35) 

Regarding dependent claims 10-12 : Marks teaches the "automatic" component 

location (CL7-42) and a system incorporating a digital camera, digitized photographs 

(picture data), and a CAD system with memory and a multiple perspective view display. 

(Abstract, Summary of Invention, Figs. 1-7, CL3-L2-10, CL4-L45, CL7-L35-67, CL7-L5- 

50) 

Independent claim 13 is drawn to: 

Generating an image of real installation model by: 
Generating picture data 

comparing component information data with picture data 
Identifying components in picture data as installation components 

Regarding independent claim 13 : As previously cited, Marks teaches a system 
and method of generating a 3D (virtual) model from an image (picture) using a 
computer system. The image is of an arrangement of physical objects (components) 
where primitives (geometric) representing a portion of a physical object (a component) 
in the image are specified as a set of parameters that correspond locations in the 
image model. (Abstract, Summary of Invention, Figs. 1-7, CL3-L2-10, CL4-L45, CL7- 
L35-67, CL7-L5-50) 

Marks does not explicitly teach comparing component information data with real 
picture data. 
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Hu teaches a system and method for identifying components by comparing 
component information data (from a knowledge base) with real picture data and 

incorporating multiple memories for storing picture data and component information. Hu 
further discloses evaluating (deriving a hypotheses via the image system) identified 
components (objects) in the picture data. (Abstract, Summary of Invention, Figs. 1-4, 6, 
8a-10b, CL13-L15-CL14-55) 

Marks mentions, but further does not explicitly teach virtual components 
- (installation). 

Crandall teaches generating a virtual model of a real system using virtual 
components (installation components) from a library of components representing the 
total virtual system (installation). (Abstract, Summary of Invention, Figs. 1a&b, 9, CL2- 
L5-11, CL4-L1-29) 

It would have been obvious to one of ordinary skill in the art at the time the 
claimed invention was made to modify the teachings of Marks relating to a system and 
method for generating a 3D (virtual) model from an image (picture), with the 
teachings ofHu relating to identifying components by comparing component 
information data (from a knowledge base) with real picture data, and to further modify 
the teachings of Marks with the teachings of Crandall relating to generating a virtual 
model of a real system using virtual components (installation components) from a 
library of components representing the total virtual system (installation), to realize a 
device and method for generating a virtual model of an installation. An obvious 
motivation exists since, as referenced by prior art, creating a 3D (virtual) model of 
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physical objects provides more efficient detection of discrepancies between the model 
and the actual image. (See Marks Background) 

Regarding dependent claims 14-16 : As also previously cited, Hu teaches image 
analysis on picture data as previously cited above. Crandall teaches using virtual 
components (installation components) from a library of components representing the 
total virtual system (installation) (CL2-L5-11) Marks teaches the manipulation of 
primitives based on geometric information (CL4-L42) and multiple window views of 
picture data and 3D (virtual) model views. (CL4-L10-25, Figs. 2-7). 

Regarding dependent claims 17-24 : Marks discloses building a 3D (virtual) model 
using the well-known techniques of "dropping", "clicking and dragging", and "rubber 
banding" in the manipulation (matching) of primitives based on geometric information 
(CL4-L42). Marks also teaches the evaluation of structural components (by function) 
to assign primitives (add components) in a 3d (virtual) installation model. (Figs. 9-25, 
CL3-L29-43, CL5-L12-25, CL6-L10-35) It further would have been obvious, and 
necessary, to "select" and "drag" installation components since all modern GUI based 
CAD system operate in this manner. Marks further teaches "automatic" component 
location (CL7-42) and a system incorporating a digital camera, digitized photographs 
(picture data), and a CAD system with memory and a multiple perspective view display. 
(Abstract, Summary of Invention, Figs. 1-7, CL3-L2-10, CL4-L45, CL7-L35-67, CL7-L5- 
50) 
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Regarding claims 25 and 26 : Claims 25 and 26 merely claim the virtual model of 
a facility that includes the same limitations as disclosed in claims 1-24. Claims 25 and 
26 are therefore rejected using the same reasoning as previously cited above. 

Conclusion 

5. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1. 136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure, careful consideration should be given prior to applicant's 
response to this Office Action. 
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U.S. Patent 5,988,862 issued Kacyra et al teaches component modeling from picture 



U.S. Patent 5,894,310 issued to Arsenault et al teaches virtual modeling of systems. 
U.S. Patent 5,812,394 issued to Lewis et al teaches virtual components and installation. 
U.S. Patent 4,937,768 issued to Carver et al teaches virtual system modeling. 



Any inquiry concerning this communication or earlier communications from the 

examiner should be directed to Fred Ferris whose telephone number is 571-272-3778 

and whose normal working hours are 8:30am to 5:00pm Monday to Friday. Any inquiry 

of a general nature relating to the status of this application should be directed to the 

group receptionist whose telephone number is 571-272-3700. If attempts to reach the 

examiner by telephone are unsuccessful, the examiner's supervisor, Jean Homere can 

be reached at 571-272-3780. The Official Fax Number is: (703) 872-9306 

5W Patent Examiner 
Simulation and Emulation, Art Unit 2128 
U.S. Patent and Trademark Office 
Randolph Building, Room 5D19 
401 Dulany Street 
Alexandria, VA 22313 
Phone: (571-272-3778) 
Fred.Ferris@uspto.gov 
January 10, 2005 
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